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LONG-TERM  GOALS 


For  effective  long-term  passive  acoustic  monitoring  of  today’s  large  data  sets,  automated  algorithms 
must  provide  the  ability  to  detect  and  classify  marine  mammal  vocalizations  and  ultimately,  in  some 
cases,  provide  data  for  estimating  the  population  density  of  the  species  present.  In  recent  years, 
researchers  have  developed  a  number  of  algorithms  for  detecting  calls  and  classifying  them  to  species 
or  species  group  (such  as  beaked  whales).  Algorithms  must  be  robust  in  real  ocean  environments 
where  non-Gaussian  and  non-stationary  noise  sources,  especially  vocalizations  from  similar  species, 
pose  significant  challenges.  In  this  project,  we  are  developing  improved  methods  for  detection, 
classification,  and  localization  of  many  types  of  marine  mammal  sounds. 

OBJECTIVES 

We  are  developing  advanced  real-time  passive  acoustic  marine  mammal  detection,  classification,  and 
localization  methods  using  a  two-pronged  approach:  developing  improved  DCL  algorithms,  and 
developing  standardized  interfaces  and  software. 

First,  we  are  developing,  testing,  and  characterizing  advanced  DCL  algorithms: 

1 .  Echolocation  click  classification.  Algorithms  are  being  developed  and  tested  for  several  species 
of  beaked  whales  and  small  odontocetes. 

2.  Tonal  signal  detection  and  classification.  Algorithms  are  being  tested  for  several  species  of 
mysticetes  and  for  small  odontocetes. 

3.  Multi-sensor  localization.  Algorithms  will  be  developed  and  tested  on  datasets  containing 
sounds  of  both  odontocetes  and  mysticetes. 

Second,  improved  DCL  software  will  be  developed  and  both  existing  and  new  methods  will  be  made 
available  to  users.  The  key  contribution  will  be  the  development  of  four  well-specified  interfaces,  for 
detection,  feature  extraction,  classification,  and  localization. 

APPROACH 

Odontocete  click  detection  and  classification.  A  multi-class  support  vector  machine  (SVM)  classifier 
was  previously  developed  (Jarvis  et  al.  2008).  This  classifier  both  detects  and  classifies  echolocation 
clicks  from  five  species  of  odontocetes,  including  Blainville’s  and  Cuvier’s  beaked  whales,  Risso’s 
dolphins,  short-finned  pilot  whales,  and  spenn  whales.  Here  Moretti’s  group,  particularly  S.  Jarvis,  is 
improving  the  SVM  classifier  by  resolving  confusion  between  species  whose  clicks  overlap  in 
frequency. 

The  current  real-time  system  of  Roch  et  al.  for  odontocete  click  classification  is  based  on  Gaussian 
mixture  models  (GMMs)  using  cepstral  feature  vectors.  Cepstral  feature  vectors  provide  a  compact 
representation  of  the  spectrum  (Rabiner  and  Juang  1993)  that  let  the  system  represent  echolocation 
spectra  using  a  reduced  number  of  coefficients,  providing  a  lower-dimensional  feature  space  than  using 
a  standard  representation  of  the  spectrum.  This  system  will  be  extended  to  cover  more  species  and 
more  recording/noise  environments.  In  a  separate  project,  Roch  is  working  with  personnel  at  Univ. 
Calif.  San  Diego  on  developing  new  features  based  on  subspace  models  and  improved  noise 
compensation.  The  subspace  models  use  hierarchical  principal  components  analysis  and  random- 
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projection  trees  (Freund  et  al.  2007)  to  learn  new  feature  sets  that  will  be  used  in  place  of  cepstral 
feature  vectors.  The  noise  modeling  will  examine  how  to  more  effectively  estimate  background  noise 
and  compensate  for  it,  taking  into  account  interactions  between  noise  and  source  (Ross  1976). 

Tonal  signal  detection  and  classification.  “Tonal  signal”  is  a  generic  term  for  frequency-modulated 
calls  such  as  mysticete  moans  or  odontocete  whistles.  Methods  for  detecting  and  classifying  such 
sounds  are  being  developed  and  applied  to  both  odontocete  whistles  and  baleen  whale  vocalizations, 
including  minke  (Balaenoptera  acutorostrata),  blue  (B.  musculus),  and  humpback  ( Megaptera 
novaeangliae )  whales. 

Odontocete  click  removal.  The  methods  being  developed  here  classify  (to  the  species  level)  the 
odontocete  whistles  that  are  extracted  automatically  via  the  Silbido  tonal  contour  following  system 
(Roch  et  al.  2010).  Research  led  by  Roch  focuses  on  the  areas  of  signal  processing  and  Silbido' s  search 
algorithm  to  further  refine  this  algorithm.  Echolocation  clicks  result  in  broad-band  energy  producing 
interfering  peaks  in  the  time-frequency  domain.  These  are  being  mitigated  by  locating  echolocation 
clicks  through  an  existing  detection  algorithm  (Soldevilla  et  al.  2008,  Roch  et  al.  201 1)  based  on 
Teager  energy  (Kaiser  1990,  Kandia  and  Stylianou  2006),  and  then  removing  it  by  interpolation. 

In  observing  expert  analysts  classify  whistles  to  species,  we  have  noted  that  experts  tend  to  comment 
on  the  general  shape  of  a  whistle.  Extracted  contours  will  be  classified  to  species  using  hidden  Markov 
models  (HMMs)  which  are  capable  of  modeling  temporal  transitions,  thus  exploiting  the  shape.  HMMs 
have  been  used  previously  to  classify  signature  whistles  to  groups,  but  a  general  approach  requires 
more  general  models  that  can  capture  inter-specific  variation.  We  propose  segmenting  whistles  into 
components  based  upon  easily  identifiable  landmarks  (e.g.  inflection  points),  and  creating  multiple 
models  for  components  based  upon  cluster  analysis. 

Baleen  whale  vocalizations.  Methods  developed  here  for  baleen  whale  detection  and  classification  are 
based  on  automated  detection  and  classification  of  minke  whale  ‘boing’  vocalizations  using  tonal 
signal  methods  which  have  been  previously  applied  to  US  Navy  hydrophone  data  at  PMRF  (Mellinger 
et  al.  2011;  Martin  et  al.  2013).  The  dominant  spectral  component  (DSC,  described  in  Martin  et  al. 
2013)  is  used  for  detection  of  the  call,  as  this  component  is  typically  the  last  component  detected  at 
long  ranges  (>  30  km).  Minke  boing  call  detection  is  used  here  has  a  first-stage  detection  step  similar 
to  the  tonal  detection  processing  described  by  Mellinger  et  al.  (2011).  A  second  stage  is  also  used 
which  processes  a  frequency  band  from  1320  to  1450  Hz  to  detect  the  onset  frequency-modulation 
(FM)  upsweep  component  of  the  call  to  obtain  a  more  accurate  estimate  of  the  start  time  of  the  call  for 
later  localization  processing.  Then  a  third  stage  calculates  the  frequency  with  high  spectral  resolution 
(0.72  Hz  per  bin)  of  the  DSC  for  each  detected  boing  to  help  associate  calls  from  individuals  and  in 
some  cases  to  help  track  individuals  over  multiple  hours. 

Advanced  localization  algorithms.  The  first  requirement  for  passive  acoustic  localization  of  marine 
mammals  is  the  need  to  associate  the  detection  of  an  individual  signal  as  it  is  received  across  the  array 
of  widely  spaced  hydrophones.  Moretti  is  leading  the  effort  to  develop  a  nearest-neighbor  approach  to 
detection  association.  This  approach  still  uses  TDOA/hyperbolic  methods,  but  will  not  discard  TDOA 
from  pairs  of  detections  when  the  nonnally  requisite  3  detections  are  not  achieved.  Rather,  detections 
from  a  given  hydrophone  will  be  associated  with  detections  from  all  of  its  nearest  neighbors  and  pair¬ 
wise  TDOAs  will  be  calculated. 
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Software  and  interfaces.  An  Application  Programming  Interface  (API)  is  a  specification  of  a  set  of 
procedure  calls  (for  objects,  methods),  data  types  (scalars,  structures,  classes,  etc.),  and  protocols  for 
use  of  the  procedures  and  data  types,  making  it  relatively  simple  for  a  developer  to  add  new  algorithms 
to  an  existing  system.  Ishmael’s  (Mellinger  2001)  interfaces  for  detection  and  localization  comprise  a 
relatively  complex  set  of  object  class  methods  (procedure  calls)  and  data  types;  although  it  is 
standardized,  it  is  hardly  straightforward  or  well-documented.  The  M3R  system  (Morrissey  et  al.  2006) 
has  a  format  for  standardized  data  serving  and  detection  message  passing.  We  are  developing  and 
testing  APIs  for  these  systems. 

WORK  COMPLETED 

Meetings,  data  sharing  site,  and  funding: 

(1)  We  have  had  teleconference  meetings  approximately  monthly  to  discuss  both  technical  details  and 
project  logistics,  with  a  face-to-face  meeting  on  3 -Nov-20 13  at  the  ASA  meeting. 

(2)  We  established  a  private  Internet-accessible  site  for  sharing  data  sets,  meeting  minutes,  and  code. 
The  site  is  private  since  some  of  the  data,  while  not  classified,  is  considered  sensitive. 

(3)  Funding  for  the  first  three  years  of  the  project  reached  all  project  members.  Year-4  funding  is 
working  its  way  through  financial  systems  to  reach  project  members  OSU  and  SDSU. 

RESULTS 

Detection/classification  algorithms:  tonal  sounds.  Advanced  automated  detection/classification 
methods  have  been  developed  and  applied  to  fin,  sei,  Bryde’s,  minke,  and  humpback  whales  (Fig.  1). 
The  minke  algorithm  includes  detection  of  minke  boing  calls,  while  the  other  detections  are  more 
generic:  for  humpback  song  units  between  200  Hz  and  1200  Hz,  and  for  fin,  sei,  and  Bryde’s  whale 
calls  between  approximately  15  Hz  and  50  Hz.  Humpback  whale  song  unit  processing  is  also  done 
using  the  Generalized  Power  Law  (GPL)  detector  (Helble  2012).  Improvements  include  a  common 
parallel-processing  front  end. 

Another  approach  to  whistle  classification  has  concentrated  on  increasing  the  purity  of  the 
automatically  generated  whistle  clusters  prior  to  training  hidden  Markov  models.  We  also  completed 
work  on  exploiting  ridge  information  in  spectrograms  to  help  identify  delphinid  whistles 
(Kershenbaum  and  Roch,  2013).  By  looking  at  a  spectrogram  as  a  topological  map,  it  is  possible  to 
examine  the  direction  in  gradient  vectors  and  look  for  coherent  regions  where  the  signs  of  the  gradient 
vectors  swap.  This  algorithm  has  been  incorporated  into  our  whistle  extraction  algorithm  Silbido  (Roch 
et  al.,  2011a).  Work  on  unsupervised  clustering  of  whistles  was  also  refined.  The  techniques  have  been 
evaluated  on  spinner  ( Stenella  longirostris),  common  ( Delpinus  spp.),  and  bottlenose  dolphins 
( Tursiops  truncatus );  all  show  potential  for  stable  unsupervised  whistle  component  clustering  (Fig.  2). 

Detection/classification  algorithms:  odontocete  clicks.  An  iterative  nonnalized  least  mean  squares 
(NLMS)  method  and  a  subspace-based  method  -  to  separate  a  raw  audio  stream  into  ‘noise’  and 
‘signal’  components  -  were  developed  for  noise  reduction  in  ocean  recordings.  Although  the  methods 
are  not  capable  of  completely  separating  clicks  from  noise,  they  do  improve  the  signal-to-noise  ratio  of 
the  clicks  and  help  improve  detection  performance.  A  new  detector  was  developed  using  NLMS  and 
the  noise-subspace  method  combined  with  the  existing  energy  ratio  mapping  algorithm  (ERMA) 
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detector  (Klinck  and  Mellinger  2011).  This  combination  improved  overall  performance  at  detecting 
clicks  of  Blainville’s  beaked  whales  in  noise.  A  paper  on  this  was  written  (Lu  et  al.  in  prep.). 

Blainville’s  beaked  whales  generate  homing  pulses,  tenned  buzzes,  prior  to  prey  capture  attempts. 

Buzz  clicks  not  only  are  produced  an  a  significantly  faster  inter-click  interval  (ICIbUzz<<  ICIforage),  but 
the  structure  of  the  click  is  also  different  (Jarvis  et  al.  2008).  A  class-specific  support  vector  machine 
(CS-SVM)  classifier  (Jarvis  2012)  was  developed  specifically  for  Md  buzz  clicks.  However,  since 
buzzes  have  a  much  lower  source  level  than  foraging  clicks,  the  detection  threshold  of  the  buzz 
classifier  must  be  set  correspondingly  lower.  Running  the  buzz  class,  with  its  very  low  threshold, 
continuously  can  cause  an  unacceptably  high  number  of  false  alarms.  To  avoid  this  problem,  the  buzz 
classifier  is  launched  only  after  a  Md  foraging  click-train  has  been  detected  (i.e.,  80%  of  all  the  clicks 
detected  in  the  past  20  s  have  been  classified  as  Md  foraging  clicks).  Then  the  buzz  classifier  runs  for 
only  30  minutes,  the  average  vocal  period  of  an  Md  dive.  See  Figs.  3-5. 

Localization  and  tracking.  Humpback  song  unit  automated  localizations  are  being  investigated  via 
cross  correlation  of  GPL  outputs,  rather  than  spectrograms  or  raw  time  series,  and  are  showing 
promise.  GPL  processing  appears  to  work  in  the  presence  of  Navy  MFAS  activity. 

Two  model-based  localization  methods  have  been  developed  for  baleen  whale  localization.  One  uses 
detection  start  times  and  associations  to  determine  time  difference  of  arrivals,  while  the  second  uses 
cross-correlation  of  call  sequences  between  hydrophone  pairs.  The  Matlab  implementation  of  the  first 
method  was  initially  pursued  on  this  effort  with  new  capabilities  (e.g.,  depth  estimation)  and 
refinements  being  done  in  collaboration  with  an  LMR  project  (see  Related  Projects,  below).  By 
automatically  associating  sequential  localizations  to  individual  whales,  additional  infonnation  on  the 
species  is  available  such  as  acoustic  ecology  (e.g.  call  rates)  and  kinematic  behavior  (e.g.  speed,  depth, 
heading  rates).  This  may  be  termed  ‘tracking’  of  individuals  and  serves  to  automate  what  humans  do 
visually  when  presented  time  sequences  (time)  of  whale  localizations  presented  on  nautical  charts 
(space). 

For  minke,  fin,  sei,  Bryde’s,  and  humpback  whales,  rudimentary  kinematic  tracking  of  individual 
whales  enabled  automatic  call  interval  analysis.  Call  intervals  automatically  obtained  not  only  helped 
confirm  the  species,  but  also  added  infonnation  on  the  species’  call  rates  and  behavior. 

Last  year  a  number  of  advances  were  made  on  localization  of  beaked  whales  using  time  differences  of 
arrival  (TDOAs)  gained  from  widely  spaced  sensors  (Baggenstoss  in  press).  A  difficulty  that  occurs  in 
localization  is  the  large  number  of  ambiguous  solutions  (Fig.  6),  arising  from  false  time-delay 
measurements  arise  when  the  TDOA  is  measured  between  clicks  or  click-trains  from  different  sources, 
different  propagation  paths,  or  different  periods  of  a  given  click  train.  The  method  developed  here  is  an 
efficient  algorithm  that  can  efficiently  track  multiple  whales  in  real  time.  Our  approach  uses  a  multi¬ 
hypothesis  tracker  (MHT)  with  pruning  to  maintain  a  large  number  (thousands)  of  candidate  tracks. 
With  each  new  update,  each  track  is  updated  as  a  Kalman  filter  using  several  potential  associations, 
then  candidate  tracks  are  pruned.  Examples  are  shown  in  Figs.  7-8.  The  track  position  of  a  DTAGged 
whale  obtained  from  the  MHT  algorithm  compared  favorably  to  the  ground  truth  position  data  from 
the  tag  (Figs.  9-10). 

Software:  The  architecture  for  writing  detection,  classification,  and  localization  modules  has  been 
completed  and  communication  between  Ishmael  and  PAMGUARD  and  a  test  module  has  been 
established.  The  architecture  provides  a  translation  library  for  each  DCL  platform  supported  that 
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marshals  data  into  a  fonnat  that  can  be  shared  with  other  processes.  Modules  run  as  separate  programs 
that  share  a  limited  region  of  memory  with  the  DCL  platform.  This  allows  modules  written  on 
platforms  that  require  separate  processes  (e.g.  Matlab,  R)  to  be  gracefully  handled.  Users  designing 
classification  modules  will  configure  the  DCL  platform  to  send  data  to  their  module  and  make  calls  to 
a  standard  interface  library.  Results  are  sent  back  to  the  DCL  platform  in  a  similar  manner.  The  plug-in 
architecture  has  been  demonstrated  with  pass  through  (identity  function)  modules  in  Java  and  C. 
Ongoing  work  is  developing  interfaces  to  handle  event  processing  such  as  passing  detections  or 
localizations  to  PAMGUARD/Ishmael  for  further  processing. 

IMPACT/APPLICATIONS 

For  the  Navy,  passive  acoustic  monitoring  (PAM)  provides  a  means  of  long-term  monitoring  of  many 
cetacean  populations,  especially  over  areas  of  high  interest.  Such  areas  are  repeatedly  subjected  to 
Navy  exercises  involving  intense  sounds,  especially  multi-ship  mid-frequency  active  (MFA)  sonar. 
Currently,  required  environmental  monitoring  is  dependent  primarily  on  visual  line  transect  surveys 
that  are  costly  and,  in  the  case  of  aerial  surveys,  significantly  dangerous.  In  both  the  areas  critical  to 
the  Navy  and  in  other  areas  critical  to  marine  mammals,  PAM  is  dependent  on  automated  DCL 
methods.  The  advanced  DCL  algorithms  being  developed  here  will  make  PAM  more  effective  and 
efficient;  the  algorithm  implementations  across  standardized  interfaces  that  handle  both  real-time  and 
pre-recorded  data  streams  from  diverse  platforms  will  make  them  available  to  Navy  fleet  operators  as 
well  as  the  wider  marine  mammal  research  community. 

TRANSITIONS 

Both  model-based  localization  algorithms  have  been  transitioned  to  an  Operations  and  Maintenance 
Navy  (OMN)-funded  effort  monitoring  for  marine  mammals  in  the  Hawaii  Range  Complex  (see 
Related  Projects). 

The  automated  methods  for  detecting  and  localizing  baleen  whales  in  this  effort  have  transitioned  to 
funded  efforts  monitoring  for  marine  mammals  during  U.S.  Navy  training  activity  at  the  Pacific 
Missile  Range  Facility  in  the  Hawaii  Range  Complex.  (Martin  et  al.  2014  in  prep). 

RELATED  PROJECTS 

“Passive  Autonomous  Acoustic  Monitoring  of  Marine  Mammals  with  Seagliders”  (N00014-10-1- 
0387)  award  to  Mellinger  (and  Klinck).  The  methods  developed  here  are  likely  to  be  implemented  in 
the  Seaglider  acoustic  system  for  real-time  detection  and  classification  of  marine  mammal  sounds. 

“Acoustic  Metadata  Management  and  Transparent  Access  to  Networked  Oceanographic  Data  Sets” 
(NOPP  N000 14-1 1-1-0697)  award  to  PI  Marie  Roch,  Co-PI  Simone  Baumann-Pickering,  John  A. 
Hildebrand,  et  al.  A  metadata  management  system  is  being  developed,  which  allows  access  to  locally 
stored  acoustic  detections  and  metadata  and  links  in  a  standardized  way  to  external  sources,  such  as 
oceanographic  or  ephemeris  data.  We  will  design  our  DCL  plugins  to  provide  outputs  that  can  easily 
be  stored  in  the  acoustic  metadata  database. 

“PMRF  acoustic  data  collection  and  analysis”  (N000701WR4C673)  OMN  funded  effort  supporting 
COMPACFLT’s  Hawaii  Range  Complex  monitoring  of  marine  mammals  during  training  activities. 
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Minke  and  Fin/Sei/Bryde's  presence  at  PMRF 
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Figure  1.  Minke,  fin,  sei,  and  Bryde’s  whales  automatically  localized  detections  per  hour  for  3366  h 
of  data  from  Jan  2011  to  Feb  2014.  Strong  seasonal  presence  for  the  majority  of  the  data  with 
suggestions  that  minke  are  more  abundant  due  to  the  higher  counts.  Data  for  Aug  6,  2013  has  been 
confirmed  to  be  similar  calls  to  those  reported  for  Bryde  ’s.  Zero  calls  shown  as  0.01  per  h  for  the 

semi-log  scale  plot. 
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Figure  2.  Visualization  of  whistle  component  clusters  for  spinner  (upper)  and  common  dolphins 
(lower).  These  clusters  were  automatically  generated  without  the  need  to  specify  the  number  of 
possible  clusters.  The  graphs  show  the  association  between  nodes  with  longer  connecting  edges 
(lines)  indicating  higher  dissimilarity.  Nodes  are  color  coded  to  display  the  automatically  learned 
categories,  and  the  grid  next  to  each  graph  shows  the  whistles  associated  with  each  color.  In  each 
case,  some  whistle  components  (black)  were  not  clustered. 
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Class 


CS-SVM  Results:  spc-20140708-075245-5a6-H64  -  Day  189  17:45:54  to  19:12:54Z 


• 

Md  =  0.772 

* 

Zc  =  0,174 
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Md  --  buzz 

Zc  --  buzz 

Figure  3:  Md  buzz  detected  within  the  context  of  a  foraging  dive.  Blue  dots  represent  clicks 
classified  as  Md  foraging  clicks  and  blue  x's  are  clicks  that  were  classified  as  buzz  clicks.  Green  dots 

are  clicks  that  were  (erroneously)  classified  as  Zc. 
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Figure  4:  The  output  of  the  CS-SVM  classifier  for  a  full  Md  dive  recorded  at  AUTEC.  Several 

buzzes  were  detected. 
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Class 


CS-SVM  Results:  spc-20140708-075245-6-H64  -  Day  189  18:27:51  to  19:12:54Z 


Figure  5:  Close  up  of  buzz  detections  from  figure  2.  Often,  the  whole  buzz  is  not  detected  probably 
due  to  rapid  head  motion  of  the  animal  as  it  follows  its  prey  just  prior  to  capture. 
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Figure  6:  Example  of  tracking  highly  ambiguous  localizations. 
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Figure  7:  Output  of  MHT  -  Kalman  filtering  algorithm  tracking  for  2  Blainville's  beaked  whales 
(blue  and  red  solid  lines)  using  bottom-mounted  hydrophones  at  AUT EC  (cyan  dots  are  raw 

localizations). 
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Figure  8:  Depth  tracks  (blue  and  red  solid  lines)  from  the  2  Blaineville 's  beaked  whales  in  Figure  2. 

Cyan  dots  are  raw  depths. 
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Figure  9:  X-Y  track  from  the  MFIT  algorithm  (solid  blue  line)  for  a  segment  of  a  Blainville's  beaked 
whale  dived  recorded  on  a  DTAG.  The  red  line  represents  the  ground  truth  position  recorded  by  the 

tag  and  the  cyan  dots  are  the  raw  localizations. 
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Figure  10:  Depth  track  for  the  whale  in  figure  4.  Solid  blue  line  represents  the  output  of  the 
MHT  tracker,  the  red  line  represents  the  ground  truth  depth  recorded  by  the  tag  and  the 

cyan  dots  are  the  raw  depths. 
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